Hybrid Layered Video Encoding for Mobile Internet-Based Computer Vision and Multimedia Applications

Mobile networked environments are typically resource constrained in terms of the available bandwidth and battery capacity on mobile devices. Realtime video applications entail the analysis, storage, transmission, and rendering of video data, and are hence resource-intensive. Since the available bandwidth in the mobile Internet is constantly changing, and the battery life of a mobile video application decreases with time, it is desirable to have a video representation scheme that adapts dynamically to the available resources. A Hybrid Layered Video (HLV) encoding scheme is proposed, which comprises of content-aware, multi-layer encoding of texture and a generative sketch-based representation of the object outlines. Different combinations of the textureand sketch-based representations result in distinct video states, each with a characteristic bandwidth and power consumption profile. The proposed HLV encoding scheme is shown to be effective for mobile Internet-based multimedia applications such as background subtraction, face detection, face tracking and face recognition on resource-constrained mobile devices.